1. Field of the Invention
The present invention relates to managing resources in a compute environment and more specifically to a system and method of querying and controlling resources in a compute environment such as a multi-cluster environment.
2. Introduction
Grid computing may be defined as coordinated resource sharing and problem solving in dynamic, multi-institutional collaborations. Many computing projects require much more computational power and resources than a single computer may provide. Networked computers with peripheral resources such as printers, scanners, I/O devices, storage disks, scientific devices and instruments, etc. may need to be coordinated and utilized to complete a task.
Grid/cluster resource management generally describes the process of identifying requirements, matching resources to applications, allocating those resources, and scheduling and monitoring grid resources over time in order to run grid applications as efficiently as possible. Each job submitted for processing will utilize a different set of resources and thus is typically unique. In addition to the challenge of allocating resources for a particular job, grid administrators also have difficulty obtaining a clear understanding of the resources available, the current status of the grid and available resources, and real-time competing needs of various users. General background information on clusters and grids may be found in several publications. See, e.g., Grid Resource Management, State of the Art and Future Trends, Jarek Nabrzyski, Jennifer M. Schopf, and Jan Weglarz, Kluwer Academic Publishers, 2004; and Beowulf Cluster Computing with Linux, edited by William Gropp, Ewing Lusk, and Thomas Sterling, Massachusetts Institute of Technology, 2003.
It is generally understood herein that the terms grid and cluster are interchangeable in that there is no specific definition of either. The definition of a grid is very flexible and may mean a number of different configurations of computers. The introduction here is meant to be general given the variety of configurations that are possible. In general, a grid will comprise a plurality of clusters as is shown in FIG. 1. Several general challenges exist when attempting to maximize resources in a grid. First, there are typically multiple layers of grid and cluster schedulers. A grid 100 generally comprises a group of clusters 110, 112 or a group of networked computers. A grid scheduler 102 communicates with a plurality of cluster schedulers 104A, 104B and 104C. Each of these cluster schedulers communicates with a plurality of resource managers 106A, 106E and 106C. Each resource manager communicates with a series of compute resources shown as nodes 108A, 108B, 108C within cluster 110 and 108D, 108E, 108F in cluster 112.
There are various vendors of resource managers 106A, 106B, 106C that require differing means of communication to and from the resource manager. For example, one resource manager vendor may have software that only communicates with certain types of compute resources, such as MicroSoft or Linux operating systems, hard drives using certain protocols and so forth. This can cause challenges when a cluster includes a variety of cluster resource managers in order to communicate with these differing products.
Other challenges of the model shown in FIG. 1 exist as well. Local schedulers (which may refer to either the cluster schedulers 104 or the resource managers 106) are closer to the specific resources 108 and may not allow grid schedulers 102 direct access to the resources. Examples of compute resources include data storage devices such as hard drives and computer processors. The grid level scheduler 102 typically does not own or control the actual resources. Therefore, jobs are submitted from the high level grid-scheduler 102 to a local set of resources with no more permissions that then user would have. This reduces efficiencies.
Another issue that reduces efficiency is the heterogeneous nature of the shared resources. Without dedicated access to a resource, the grid level scheduler 102 is challenged with the high degree of variance and unpredictability in the capacity of the resources available for use. Most resources are shared among users and projects and each project varies from the other.
Furthermore, the performance goals for projects differ. Grid resources are used to improve performance of an application but the resource owners and users have different performance goals: from optimizing the performance for a single application to getting the best system throughput or minimizing response time. Local policies may also play a role in performance.
A bottleneck in the middleware component of the cluster or grid environment is the difficulty in communication and managing the diverse compute resources. If the clusters 110 and 112 include a group of compute resources that have diverse communication protocols and requirements across different resource managers 106A, 106B, 106C, then a cluster manager or company establishing or provisioning a cluster may need to purchase or license multiple resource managers to communication and assign jobs to all the compute resources.
FIG. 2 illustrates the prior art in more detail. Typically the cluster scheduler 104A communicates with only one resource manager server 106A. The resource manager server 106A retrieves data from a number of resource manager clients 120A, 120B, 120C that are located on each of the compute hosts 122A, 122B, 122C within a respective cluster node 108A, 108B, 108C. The clients 120A, 120B, 120C obtain information regarding the state of the node 108A, 108B, 108C, the load on the node, the configuration of the node, and similar properties.
This model is acceptable for simple clusters but it has a number of problems. A primary problem in this scenario is that the cluster scheduler's view of the compute resources is bound by the resource manager server 106A and it is not able to obtain any information that is not contained within the resource manager 106A. The cluster scheduler 104A is not able to take any action that is not within the scope of the communication and control capabilities of the resource manager 106A. For an end user or a customer site, they are bound to select a single resource manager and are unable to pick and choose one resource manager combined with the features of another. The standard resource manager 106A is designed on a locked-in model and purchasers must simply find the best resource manager with the combination of features and purchase it or look to purchasing and incorporating multiple and diverse resource managers to meet all their needs.
What is needed in the art is a way of allowing a scheduler to contact multiple sources of information and multiple sources of control so that an end user site can pick and choose which groupings of services to utilize from multiple sources. What is further needed in the art is an improved method and system for managing the disparate compute resources at a single location within the context of a cluster environment where there are a variety of cluster resource managers and a variety of compute resources.